Integrated medical imaging and surgical robotic system

ABSTRACT

Methods and systems for performing robotically-assisted surgery in conjunction with intra-operative imaging. A method includes moving a robotic arm with respect to a patient and an imaging device to move an end effector of the robotic arm to a pre-determined position and orientation with respect to the patient based on imaging data of the patient obtained by the imaging device. The robotic arm maintains the end effector in the pre-determined position and orientation with respect to the patient and does not collide with the imaging device or with the patient when the imaging device moves with respect to the patient.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of application Ser. No. 15/130,258filed on Apr. 15, 2016, which claims benefit of U.S. ProvisionalApplication No. 62/147,924, filed on Apr. 15, 2015, the entire contentsof which are incorporated herein by reference.

BACKGROUND OF THE INVENTION

Surgical procedures, such as minimally-invasive procedures, may requirea surgeon to insert surgical tools inside the body of the patient to aparticular depth to reach the target area inside the patient's body.This can be a challenging and time consuming process. Surgeons mayutilize pre-operative diagnostic images (e.g., x-ray CT, MM) to helpguide the surgical tools to the correct location and depth in the bodywithout damaging other tissue or organs of the patient. However, thereis still the possibility of inaccuracies in the insertion and placementof the tools, which may result in tools being guided to an incorrectposition in the body and/or causing injury or damage to other portionsof the patient's anatomy. To avoid such problems, a surgeon may opt toperform a more invasive procedure than might otherwise be necessary.However, this may substantially increase the time and cost of theprocedure, and may also increase blood loss, pain and recovery time forthe patient.

BRIEF SUMMARY OF THE INVENTION

Various embodiments include methods and systems for performingrobotically-assisted surgery. In one embodiment, a method of performingrobotically-assisted surgery includes moving a robotic arm with respectto a patient and an imaging device to move an end effector of therobotic arm to a pre-determined position and orientation with respect tothe patient based on IMAGING data of the patient obtained by the imagingdevice, where the robotic arm maintains the end effector in thepre-determined position and orientation with respect to the patient anddoes not collide with the imaging device or with the patient when theimaging device moves with respect to the patient.

Embodiments may further include determining that the imaging device ismoving with respect to the patient and moving the robotic arm tomaintain the end effector in the pre-determined position and orientationwith respect to the patient while preventing the robotic arm fromcolliding with the imaging device or with the patient while the imagingdevice moves with respect to the patient.

Further embodiments include systems for performing robotically-assistedsurgery that include a patient support, an imaging device that ismovable with respect to the patient support to obtain imaging data of apatient positioned on the patient support, a robotic arm configured tomove an end effector of the robotic arm to a pre-determined position andorientation with respect to the patient positioned on the patientsupport based on imaging data obtained by the imaging device, and amotion tracking apparatus including a camera attached to the imagingdevice or to the patient support, where the camera is positioned totrack the position of one or more objects in a surgical area. Inembodiments, the camera may move independently of the imaging device andthe patient support to maintain the surgical area within the field ofview of the camera.

Further embodiments include a system for performing robotically-assistedsurgery that includes an x-ray imaging device including an x-ray sourceand an x-ray detector mounted to a support structure such that at leastone of the x-ray source and the x-ray detector is configured to rotatewith respect to the support structure to obtain x-ray images fromdifferent projection angles relative to an object being imaged, and arobotic arm having a first end configured to extend into an imaging areabetween the source and the detector and a second end attached to thesupport structure.

Further embodiments include a computing device including a processorconfigured with processor-executable instructions to perform operationsof the embodiment methods described above. Further embodiments include anon-transitory processor-readable storage medium having stored thereonprocessor-executable software instructions configured to cause aprocessor to perform operations of the embodiment methods describedabove. Further embodiments include a computing device that includesmeans for performing functions of the operations of the embodimentmethods described above.

BRIEF DESCRIPTION OF THE DRAWINGS

Other features and advantages of the present invention will be apparentfrom the following detailed description of the invention, taken inconjunction with the accompanying drawings of which:

FIGS. 1A-1C are perspective views showing the front (FIGS. 1A and 1C)and rear (FIG. 1B) sides of a system for performing robotically-assistedsurgery including a pair of robotic arms attached to a gantry of animaging device according to one embodiment.

FIGS. 2A-2C are perspective views showing the front (FIGS. 2A and 2C)and rear (FIG. 2B) sides of the system of FIGS. 1A-1C with the gantrytranslated to the surgical area of the patient.

FIG. 3 is a system block diagram that schematically illustrates variouscomponents of a system for performing robotically-assisted surgeryaccording to one embodiment.

FIG. 4 schematically illustrates a method of defining a trajectory forinsertion of a surgical tool into a patient's body using image data.

FIG. 5 is a process flow diagram illustrating a method for operating arobotic arm to perform robotically-assisted surgery according to anembodiment.

FIGS. 6A-6C are perspective views of a system for performingrobotically-assisted surgery having a robotic arm attached to a side ofan imaging gantry facing away from the patient.

7A-7C show the system of FIGS. 6A-6C with the gantry translatedpartially along the length of the patient.

FIGS. 8A-8D illustrate a system for performing robotically-assistedsurgery including a pair of robotic arms attached to a patient supportwith a gantry of an imaging system translated to the surgical area ofthe patient.

FIGS. 9A-9B illustrate the system of FIGS. 8A-8B with the gantrytranslated away from the surgical area.

FIGS. 10A-10C illustrate a system for performing robotically-assistedsurgery including a robotic arm and a camera for a motion trackingapparatus attached to a patient support for a patient in a sittingposition.

FIGS. 11A-11C illustrate the system of FIGS. 10A-10C with a gantry of animaging device translated to the surgical area of the patient.

FIGS. 12A-12B illustrate the system of FIGS. 10A-10C and 11A-11C withthe patient support rotated with respect to the imaging device.

FIGS. 13A-13D illustrate a system for performing robotically-assistedsurgery including a robotic arm attached to a gimbal of an imagingsystem via a support member.

FIGS. 14A-14C are side, perspective and overhead views illustrating asystem for performing robotically-assisted surgery including a roboticarm attached to an imaging system having a cantilevered O-shaped imaginggantry.

FIGS. 15A-15D are side, front perspective, rear perspective and overheadviews illustrating a system for performing robotically-assisted surgeryincluding a robotic arm attached to a C-arm imaging system.

FIG. 16 schematically illustrate a computing device which may be usedfor performing various embodiments.

DETAILED DESCRIPTION

The various embodiments will be described in detail with reference tothe accompanying drawings. Wherever possible, the same reference numberswill be used throughout the drawings to refer to the same or like parts.References made to particular examples and implementations are forillustrative purposes, and are not intended to limit the scope of theinvention or the claims.

Various embodiments are directed to an integrated system for performingrobotically-assisted surgery in conjunction with intra-operativeimaging. In recent years, there has been increased interest in the fieldof robotically-assisted surgery in which robotic systems are used to aidin surgical procedures. However, such systems are generallycharacterized by high-cost and complexity and may also be limited in thetypes of procedures they can perform. Various embodiments includesystems and methods for performing robotically-assisted surgery that maybe characterized by improved usability, workflow and ease of use. Thesystems and methods of various embodiments may be used to perform a widevariety of surgical procedures in virtually any part of a patient'sanatomy.

A system 100 for performing robotically-assisted surgery according toone embodiment is shown in FIGS. 1A-1C. FIGS. 1A and 1C are perspectiveviews showing the first (i.e., front) side of the system 100 and FIG. 1Bis a perspective view showing a second (i.e., rear) side of the system100. The system 100 includes at least one robotic arm 101 that ismovable with respect to a patient 103. In this embodiment, the system100 includes two robotic arms 101 a, 101 b that may be movedindependently of one another. It will be understood that in otherembodiments, the system 100 may include a single robotic arm or morethan two robotic arms. The movements of the robotic arm(s) 101 a, 101 bmay be controlled by a controller 105 (e.g., a computer including amemory and processor for executing software instructions) that may becoupled to the robotic arm(s) 101 a, 101 b via a wired or wireless link107. In this embodiment, the controller 105 for the robotic arms 101 a,101 b is located in a workstation/mobile cart 109 that may include adisplay 111 and one or more user input devices 100 (e.g., touchscreencontroller, keyboard, mouse, buttons, switches, etc.) to enable a userto control the operation of the system 100.

In this embodiment, each of the robotic arms 101 a, 101 b comprises amultijoint arm that includes a plurality of linkages 113 connected byjoints 115 having actuator(s) and optional encoder(s) to enable thelinkages to bend, rotate and/or translate relative to one another inresponse to control signals from the control system 105. A first end 117of the robotic arm 101 a, 101 b may be fixed to a structure 40 and asecond end 119 of the arm may be freely movable with respect to thefirst end 117. An end effector 121 is attached to the second end 119 ofthe robotic arm 101 a, 101 b. In some embodiments, the end effector 121may be an invasive surgical tool, such as a needle, a cannula, a cuttingor gripping instrument, an endoscope, etc., that may be inserted intothe body of the patient. In other embodiments, as described in furtherdetail below, the end effector 121 of the robotic arm 101 a, 101 b maybe a hollow tube or cannula that may receive an invasive surgical tool122 (see FIG. 1C), including without limitation a needle, a cannula, atool for gripping or cutting, an electrode, an implant, a radiationsource, a drug and an endoscope. The invasive surgical tool 122 may beinserted into the patient's body through the hollow tube or cannula by asurgeon. An end effector 121 comprising a hollow tube or cannula may bemade of a radiolucent material, such as a carbon-fiber or thermoplasticmaterial.

The patient 103, which may be a human or animal patient, may be locatedon a suitable patient support 60, which may be a surgical table as shownin FIGS. 1A-1C. The patient support 60 in this embodiment is raised offthe ground by a support column 50. During a surgical procedure, therobotic arms 101 a, 101 b may be located partially or completely withinthe sterile surgical field, and thus may be covered by a surgical drapeor other sterile barrier (not shown for clarity). In embodiments, theend effector 121 (e.g., a hollow tube or cannula) may be a sterilizedcomponent that may be attached (e.g., snapped into) the end 119 of therobotic arm 101 over the drape. The end effector 121 may be a sterile,single-use (i.e., disposable) component that may be removed anddiscarded after use.

The system 100 also includes an imaging device 125 that may be used toobtain diagnostic images of the patient 103. The imaging device 125 maybe located in proximity to both the patient 103 and the at least onerobotic arm 101 a, 101 b (e.g., within 10 meters, such as less than 5meters, including 0-3 meters from the patient 103 and arm(s) 101 a, 101b), and is preferably located within the same room (e.g., an operatingroom). In the embodiment of FIGS. 1A-1C, the imaging device 125 includesa base 20 and a gantry 40 located above the base 20. The gantry 40 inthis embodiment includes a substantially O-shaped housing (i.e., a ring)defining a bore 16. The gantry 40 includes one or more imagingcomponents (not shown for clarity) located within the housing of thegantry 40 that are configured to obtain image data of at least a portionof an object (e.g., patient 103) positioned within the bore 16. In thisembodiment, the first end 117 of each of the robotic arms 101 a, 101 bis attached to the imaging device 125. In particular, each of therobotic arms 101 a, 101 b are attached to a first (i.e., front) side 127of the gantry 40, although it will be understood that the robotic arms101 a, 101 b may be mounted at other portions of the imaging device 125or system 100. In embodiments, the imaging device 125 may include one ormore adaptors configured to receive a robotic arm 101 at one or morelocations on the device 125. The adaptor(s) may be molded or affixed(e.g., using fasteners or adhesive) to an outer surface of the device125. The first end 117 of a robotic arm 101 may be inserted into andsecured by the adaptor. After use, the robotic arm(s) 101 may bereleased from the adaptor and removed from the imaging device 125 fortransport and/or storage. For example, the robotic arms 101 a, 101 b maybe stored on and/or transported by the cart 109.

In embodiments, the imaging device 125 may be an x-ray computedtomography (CT) imaging device. The imaging components within the gantry40 may include an x-ray source and an x-ray detector. The x-ray sourceand optionally the detector may rotate within the gantry 40 around thebore 16 to obtain x-ray image data (e.g., raw x-ray projection data) ofan object located within the bore 16. The collected image data may beprocessed using a suitable processor (e.g., computer) to perform athree-dimensional reconstruction of the object, which may be, forexample, rendered on the display 111. Examples of x-ray CT imagingdevices that may be used according to various embodiments are describedin, for example, U.S. Pat. No. 8,118,488, U.S. Patent ApplicationPublication No. 2014/0139215, U.S. Patent Application Publication No.2014/0003572, U.S. Patent Application Publication No. 2014/0265182 andU.S. Patent Application Publication No. 2014/0275953, the entirecontents of all of which are incorporated herein by reference. It willbe understood that these embodiments are provided as illustrative,non-limiting examples of imaging systems suitable for use in the presentsystems and methods, and that the present systems and methods mayutilize various types of medical imaging devices. For example,alternatively or in addition to an x-ray CT device, the imaging device125 may be an x-ray fluoroscopic imaging device, a magnetic resonance(MR) imaging device, a positron emission tomography (PET) imagingdevice, a single-photon emission computed tomography (SPECT), anultrasound imaging device, etc.

In embodiments, the imaging system 125 may be configured to move withrespect to the patient 103. For example, at least a portion of theimaging system 125 (e.g., the gantry 40) may move with respect to thepatient 103 to obtain images of a particular region of the patient'sbody, and may also be moved away from the region to facilitate asurgical procedure being performed within the region. In the embodimentshown in FIGS. 1A-1C, the patient support 60 on which a patient 103 maybe located is secured to the base 20 of the imaging system 125 by thecolumn 50, and the gantry 40 may translate with respect to the base 20,the column 50 and the patient support 60. This is illustrated in FIGS.2A-2C, which show the gantry 40 translated on the base 20 so that aportion of the patient 103 and patient support 60 are located within thebore 16 of the gantry 40.

In the embodiment of FIGS. 1A-2C, the column 50 is located on a firstend of the base 20 and the patient support 60 attached to the column 50is cantilevered over the base 20 so that the gantry 40 may translateover substantially the entire length of the patient support 60. Thegantry 40 is supported by a gimbal 30 that includes a pair of arms 31,33that extend upwards from the base 20 and are connected to opposite sidesof the gantry 40. The gimbal 30 may include bearing surfaces that travelon rails 23 on the base 20 to provide the translation motion of thegimbal 30 and gantry 40. A drive mechanism (not shown for clarity) maydrive the translation of the gimbal 30 and gantry 40. An encoder orsimilar sensing device may determine the translation position of thegantry 40 on the base 20. In embodiments, the gantry 40 may tilt withrespect to the gimbal 30.

In some embodiments, the patient support 60 and/or the column 50 may betranslated as an alternative or in addition to translating the gantry 40of the imaging system 125. For example, the patient support 60 may betranslated with respect to the column 50, or the entire column 50 andpatient support 60 may be translated with respect to the base 20. Inthis way, the patient 103 may be moved into and out of the bore 16 ofthe gantry 40. In some embodiments, the column 50 may be configured toraise and lower the height of the patient support 60 with respect to thebase 20. The patient support 60 may also be rotatable with respect tothe base 20, either by rotating the patient support 60 on the column 50or by rotating the column 50 and patient support 60 with respect to thebase 20.

The system 100 may also include a motion tracking apparatus 129 fortracking the position of at least one of the robotic arm(s) 101 a, 101 band the imaging system 125 in three-dimensional space. The trackingapparatus 129 may also track the position of the patient 103 as well asother objects, such as the patient support 50 and/or surgical tools 122within the surgical area. Various systems and technologies exist fortracking the position (including location and/or orientation) of objectsas they move within a three-dimensional space. Such systems may includea plurality of active or passive markers fixed to the object(s) to betracked and a sensing device that detects radiation emitted by orreflected from the markers. A 3D model of the space may be constructedin software based on the signals detected by the sensing device.

In the embodiment shown in FIGS. 1A-2C, the motion tracking apparatus129 is an optically-based motion tracking apparatus that includes anoptical sensor (i.e. camera) 131 and one or more markers 133. In thisembodiment, the camera 131 is attached to the gantry 40 of the imagingdevice 125 and is oriented such that the camera 131 may look directlyinto the sterile surgical field. In other embodiments, the camera 131may be mounted to another portion of the imaging device 125 or toanother component of the system 100, such as the patient support 60, ormay be mounted to a separate support structure. An advantage of theconfiguration shown in FIGS. 1A-2C is that the camera 131 may look downdirectly into the surgical field without being blocked. In embodiments,the camera 131 may be mounted to the end of an arm 135 that may includeactuator(s) for moving the camera 131 so that the surgical field ismaintained within the camera's field of view. For example, as the gantry40 moves (e.g., translates) with respect to the patient 103, the camera131 may swivel on the arm 135 and/or the arm 135 itself may pivot, bend,extend and/or contract to maintain the surgical field within the fieldof view of the camera 131.

The markers 133 may comprise any active or passive marker that may bedetected by the camera 131. The markers 133 may be fixed to variousobjects to be tracked, such as the end effectors 121 of the robotic arms101 a, 101 b, as shown in FIG. 1C. One or more markers 133 may also beattached to surgical tools 122 to enable the position and orientation ofthe various surgical tools 122 within the surgical field to be trackedin real time during a surgical procedure. One or more markers 133 mayalso be attached to other objects, such as the patient 103, the patientsupport 60 and/or the imaging device 125. In embodiments, the markers133 may be moire pattern markers that may provide measurement data forposition and orientation using a single camera 131 using Moire PhaseTracking (MPT) technology. Each marker 133 may also include a uniqueidentifier or code that may enable different objects within the camera'sfield of view to be uniquely identified and tracked. An example of anMPT-based tracking system is available from Metria Innovation Inc. ofMilwaukee, Wisconsin.

FIG. 3 is a system block diagram that schematically illustrates variouscomponents of the system 100 according to one embodiment. As discussedabove, a first controller 105 may control the operation of one or morerobotic arms 101 a, 101 b. The first controller 105 may receive feedbackdata (indicated by arrow 201) from the robotic arm(s) 101 a, 101 bregarding the status and operation of the arms 101 a, 101 b. Thefeedback data may include sensor (e.g., encoder) data that may be usedto determine the position and orientation of each of the joints 115 ofthe robotic arms 101 a, 101 b. The first controller 105 may send controlsignals (indicated by arrow 203) to the robotic arm(s) 101 a, 101 b tocontrol the movements of the arms 101 a, 101 b.

The system 100 may also include a second controller 205 for controllingthe operation of the imaging device 125. The second controller 205 mayreceive feedback data (indicated by arrow 207) regarding the status andoperation of the imaging device 125. The feedback data may includeinformation as to the position and orientation of the imaging device125, such as the position (translation or tilt) of the gantry 40 and/orthe position of the patient support 60. The second controller 205 mayalso send control signals (indicated by arrow 209) to various componentsof the imaging device 125 to control the operation of the imaging device125, including controlling the imaging device 125 to obtain image data211 (e.g., a three-dimensional CT reconstruction) of an object locatedwithin the bore 16 of the gantry 40. The image data 211 may be displayedon a display 111.

The system 100 may also include a third controller 213 for controllingthe operation of the motion tracking apparatus 129. The third controller213 may receive data sensed by the camera 131 (indicated by arrow 215)and based on this data may determine position and/or orientation datafor each of the markers 133 within the field of view of the camera 131.Based on the determined position and/or orientation of the markers 133,a three-dimensional model 219 of various objects within the surgicalspace (e.g., the patient 103, surgical tool(s) 122, the end effector(s)121 of the robotic arm(s) 101 a, 101 b, the imaging device 125, etc.)may be generated. The third controller 213 may also send control signals(indicated by arrow 217) to the camera 131 to control the operation ofthe camera 131, such as by adjusting the camera's field of view.

The first, second and third controllers 105, 205, 213 may communicateand share various data with one another, as indicated by arrows 221, 223and 225. The sharing of data including positional data enables thecontrollers to operate in a common coordinate system. For example, theimage data 211 of the patient 103 obtained by the imaging device 125 maybe registered to the position data obtained by the motion trackingapparatus 129, as is known in the field of surgical navigation systems.The position of one or more objects tracked by the motion trackingapparatus 129 may be shown on the display 111, such as overlaying thedisplay of image data 211 from the imaging device 125. Further, thefirst controller 105 may determine the position of the robotic arm(s)101 a, 101 b with respect to the rest of the system 100 based onposition data from the motion tracking apparatus 129 and/or the imagingdevice 125.

In embodiments, each of the controllers 105, 205, 213 may compriseseparate computing devices, each including a memory and processor forperforming the various functions described herein. The separatecomputing devices may communicate with one another via a suitable datalink (e.g., Ethernet). In other embodiments, two or more of thecontrollers 105, 205, 213 may be integrated in a single computingdevice.

A system 100 as described above may be used for performing surgicalprocedures on a patient 103, which may be a human or animal patient. Forexample, the patient 103 may be provided on a patient support 60 (e.g.,a surgical table), and the imaging device 125 may be used to obtainimages of the patient 103, such as a CT scan of a particular region ofthe patient's anatomy. This may include moving the gantry 40 of theimaging device 125 (e.g., translating the gantry 40 on the base 20) sothat a region of interest of the patient 103 is located within the bore16 of the gantry 40 and operating the imaging components (e.g., x-raysource and detector) to obtain image data of the patient 103.Alternately, the patient 103 may be moved into the bore 16, such as bytranslating the patient support 60 into the bore 16 of the gantry 40.

The image data obtained by the imaging device 125 may be displayed on adisplay, such as the display 111 on the mobile cart 109 shown in FIGS.1A-2C. In embodiments, a surgeon or other clinician may interact withthe image data shown in the display 111 using a suitable userinterface/input device 110 (e.g., keyboard, mouse, touchpad, trackball,touchscreen, etc.). The clinician may be able to modify the image datadisplayed on the screen of the display 111, such as by zooming in or outof a particular region, selecting or changing the particular projectionangle(s) or slices in the case of a three-dimensional tomographicreconstruction.

In embodiments, a surgeon/clinician may also select particular points onthe displayed image using an input device. This is schematicallyillustrated in FIG. 4 , which shows a display 111 that displays an image401 (e.g., a cross-sectional slice) of a region of interest of a patient103 obtained using an imaging device 125. The surgeon/clinician mayidentify and select at least one target point 403 in the displayed image401. The target point 403 may represent an end point for the insertionof a particular surgical tool 122 into the patient's body during asurgical procedure. The surgeon/clinician may also identify and selectat least one entrance point 405 on the displayed image 401. The entrancepoint 405 may represent a point on the exterior of the patient's body(e.g., the skin) through which the surgeon will insert the particularsurgical tool 122. The target point 403 and corresponding entrance point405 thus define a unique trajectory 407 through the body of the patient103, as schematically illustrated by the dashed line in FIG. 4 . Inembodiments, the surgeon may select the entrance point 405 and thetrajectory 407 within the patient's body in order to facilitate theinsertion of the surgical tool 122 to the target point 403 whileminimizing damage to other tissue or organs of the patient 103. As alsoshown in FIG. 4 , the trajectory 407 may also be extended outside of thepatient's body to define a unique vector 409 in three-dimensional spaceextending from the selected entrance point 405, as indicated by thedashed-dotted line in FIG. 4 .

FIG. 5 is a process flow diagram that illustrates a method 500 foroperating a robotic arm 101 to perform robotically-assisted surgeryaccording to one embodiment. The method 500 may be performed using thesystem 100 described above with reference to FIGS. 1A-4 . For example,the system 100 may include at least one robotic arm 101 a, 101 b havingan end effector 121. The end effector 121 may comprise a hollow tube orcannula, as described above. Each of the robotic arms 101 a, 101 b maybe moveable with respect to a patient 103 and an imaging device 125,where at least a portion of the imaging device 125, such as a gantry 40,is moveable with respect to the patient 103 to obtain imaging data ofthe patient 103. The system 100 may also include a controller 105 forcontrolling the movements of the at least one robotic arm 101 a, 101 b.

In block 501 of method 500, the controller 105 may control the at leastone robotic arm 101 to move the end effector 121 of the robotic arm 101to a pre-determined position and orientation with respect to the patient103. The pre-determined position and orientation may be based on imagingdata obtained by the imaging system 125. For example, the imaging datamay be used to determine a unique vector 409 in three-dimensional spacecorresponding to a desired insertion trajectory 407 for a surgical tool,as described above with reference to FIG. 4 . The controller 105 of theat least one robotic arm 101 a, 101 b may translate this vector 409 intoa coordinate system used for controlling the position and movement ofthe robotic arm 101 a, 101 b based on positional information receivedfrom the imaging device 125 and/or from a motion tracking apparatus 129,as described above with reference to FIG. 3 . The controller 105 maymove the at least one first robotic arm 101 a, 101 b so that the endeffector 121 of the robotic arm 101 a, 101 b is oriented along apre-defined vector 409. For example, as shown in FIG. 1A, the endeffector 121 of a first robotic arm 101 a is oriented along a firstvector 409 a. The end effector 121 of a second robotic arm 101 b isoriented along a second vector 409 b. Each of the end effectors 121 maybe positioned adjacent to a desired entrance point 405 for a surgicaltool. A surgeon may then perform an invasive surgical procedure, whichmay include inserting one or more surgical tools through the endeffectors 121 and into the body of the patient 103. The position andorientation of the end effectors 121 may ensure that the surgical tools121 follow the desired trajectory 407 (see FIG. 4 ) through thepatient's body to reach the target area.

In embodiments, a motion tracking apparatus 129 such as described abovemay be configured to track the at least one robotic arm 101 a, 101 b toensure that the end effector(s) 121 maintain the pre-determined positionand orientation with respect to the patient 103. If an end effector 121moves from the pre-determined position and orientation (e.g., due to therobotic arm being accidentally bumped), the motion tracking apparatus129 may detect this movement and alert the surgeon or other clinician.Alternately or in addition, the motion tracking apparatus 129 may send amessage to the controller 105 of the at least one robotic arm 101 a, 101b indicating a detected deviation from the pre-determined position andorientation of the end effector 121. The controller 105 may then movethe robotic arm 101 a, 101 b to compensate for the detected deviation.In some embodiments, the motion tracking apparatus 129 may also trackthe patient 103 (e.g., where a plurality of markers 133 are placed onthe patient 103) to determine whether the patient 103 has moved relativeto the end effector 121. The motion tracking apparatus 129 may notifythe surgeon when the patient 103 moves by more than a predeterminedamount. In some embodiments, the motion tracking apparatus 129 may sendmessage(s) to the controller 105 of the robotic arms(s) 101 a, 101 bregarding detected movements of the patient 103. Such movements mayinclude, for example, motion of the patient 103 corresponding to thepatient's breathing. The controller 105 may move the robotic arm(s) 101a, 101 b to compensate for any such movement (e.g., to maintain the endeffector 121 in the same position and orientation with respect to theselected entrance point 405 on the patient's body).

During a surgical procedure, the motion tracking apparatus 129 may alsobe used to track a variety of objects, including surgical tools 122,within the surgical area. For example, as discussed above, varioussurgical tools 122 may be provided with markers 122 that enable themotion tracking system 129 to identify the tools and continually tracktheir movements in three-dimensional space. Thus, as a tool 122 isinserted through an end effector 121 and into the patient's body, themotion tracking system 129 may use the detected position of themarker(s) 133 and a known geometry of the tool 122 to determine thedepth of insertion of the tool 122 into the body. This may be displayedon the display 111 of the system 100 (e.g., overlaying the image datapreviously obtained from the imaging device 125) and may aid the surgeonin determining whether the surgical tool 122 has been inserted to thedesired depth in the patient's body.

In block 503 of method 500, the controller 105 may determine that atleast a portion of the imaging device 125 is moving with respect to thepatient 103. For example, after obtaining imaging data of the patient103, the gantry 40 of the imaging device 125 may be translated away fromthe surgical area as shown in FIGS. 1A-1C to provide easier access tothe surgical area for performing a surgical procedure. The roboticarm(s) 101 a, 101 b may then be moved to a first position as shown inFIGS. 1A-1C, with the end effector(s) 121 arranged in a pre-determinedposition and orientation with respect to the patient 103. At a latertime, the surgeon may wish to obtain additional image data of thepatient 103 (e.g., to confirm the location of a surgical tool 122 withinthe patient 103), and the gantry 40 may be translated back over thesurgical area such as shown in FIGS. 2A-2C to perform an updated imagingscan.

Alternately, following the initial imaging scan, the robotic arm(s) 101a, and 101 b may be moved into position on the patient 103 as shown inFIGS. 1A-1C while the gantry 40 is still located over the surgical area.The gantry 40 may then be moved (e.g., translated) out of the surgicalarea as shown in FIGS. 2A-2C before performing the surgical procedure.

In either case, the controller 105 may determine that at least a portionof the imaging device (e.g., the gantry 40) is moving with respect tothe patient 103 based on a signal that may be received, for example,from the imaging device 125, the motion tracking system 129 and/or froma user via a user input mechanism.

In block 505 of method 500, the controller 105 may control the at leastone robotic arm 101 a, 101 b to move a first portion of the at least onerobotic arm 101 a, 101 b while the imaging device 125 moves with respectto the patient 103 while maintaining the end effector 121 of the arm inthe pre-determined position and orientation (e.g., vector 409) withrespect to the patient 103. Thus, in an embodiment such as shown inFIGS. 1A-2C, where the first end 117 of the arm 101 is attached to theportion of the imaging device 125 that moves with respect to the patient103 (i.e., the gantry 40), the controller 105 may control the movementsof the arm 101 such that as the first end 117 of the arm moves towardsor away from the patient 103, the end effector 121 maintains itsoriginal position and orientation with respect to the patient 103.

In embodiments, the controller 105 may control the movement of the firstportion of the arm 101 a, 101 b such that the arm 101 a, 101 b does notcollide with either the imaging device 125 or the patient 103 during themovement of the arm. For example, as the imaging device 125 and roboticarms 101 a, 101 b move from the position as shown in FIGS. 1A-1C to theposition as shown in FIGS. 2A-2C, at least a portion of the arms 101 a,101 b including the end effectors 121 are located inside the bore 16 ofthe gantry 40.

The controller 105 may control the movement of each of the arms 101 a,101 b so that as the gantry 40 advances towards the patient, none of thejoints 115 of the arms 101 a, 101 b collide with the side wall or innerdiameter of the ring or with the patient 103. The controller 105 maycontrol the movement(s) of the arm(s) 101 a, 101 b in accordance with amotion planning algorithm that utilizes inverse kinematics to determinethe joint parameters of the robotic arm that maintain the position andorientation of the end effector 121 while avoiding collisions with theimaging device 125 and the patient 103.

In embodiments, the controller 105 may determine the position of each ofthe robotic arms 101 a, 101 b in relation to the gantry 40 based onposition data received from the imaging device 125 (e.g., indicating thetranslation and/or tilt position of the gantry 40 with respect to thebase 20). Alternately or in addition, the controller 105 may utilizeposition information received from the motion tracking apparatus 125. Asdiscussed above, the motion tracking system 129 may be used to constructa three-dimensional model (e.g., a CAD model) of the various objectsbeing tracked by the motion tracking apparatus 129. The sharing of databetween the robotic system, the imaging device and the motion trackingapparatus may enable these systems to operate in a common coordinatesystem.

In some embodiments, the position of the patient 103 may be definedusing a freehand technique. For example, prior to commencement of thesurgical procedure, the surgeon or other clinician may use the second(i.e., distal) end 119 of a robotic arm 101 to manually trace across theexternal surface of the patient 103, such as around the surgical area.This may be used to define a three-dimensional boundary surface in thecommon coordinate system into which no portion of the at least onerobotic arm 101 a, 101 b may enter. This technique may also be used todefine boundary surfaces corresponding to other objects and componentsof the system 100 proximate to the surgical area, such as the patientsupport 60 or portions of the imaging system 125. In other embodiments,the motion tracking apparatus 129 may be used to define the boundarysurface corresponding to the patient 103, such where a plurality ofmarkers 133 are placed in different locations on the patient 103proximate to the surgical area and are tracked by the camera 131. Thepositions of the markers 133 tracked by the motion tracking apparatus129 may be used to define a three-dimensional boundary surface intowhich the robotic arm 101 a, 101 b may not enter.

In some cases, the controller 105 of the at least one robotic arm 101 a,101 b may determine that it is not possible to move a robotic armwithout either changing the position or orientation of the end effector121 with respect to the patient 103, or some part of the arm collidingwith the imaging device 125 or the patient 103. For example, atranslation of the gantry 40 may result in the arm 101 being extendedbeyond its maximum length. In other cases, the controller 105 maydetermine that no set of joint movements are possible to avoidcollisions while maintaining the end effector in a fixed position andorientation. In such a case, the controller 105 may issue an alert thatmay be perceived by the surgeon or other clinician, and may preferablyalso send a signal to the imaging device 125 to stop the motion of thegantry 40.

As the gantry 40 moves with respect to the patient 103, the camera 131may also move to maintain the surgical area within the field-of-view ofthe camera 131. In the embodiment of FIGS. 1A-2C, for example, where thecamera 131 is attached to the gantry 40 by arm 135, the camera 131 mayinclude an actuator (e.g., a DC motor-based actuator) that causes thecamera 131 to pivot on the arm 135 to keep the camera 131 pointed downinto the surgical area while the camera 131 moves with the gantry 40.This may enable the motion tracking apparatus 129 to continually trackthe position of objects within the surgical area as the gantry 40 androbotic arms 101 a, 101 b move. Thus, the motion tracking apparatus 129may provide a redundant safety feature in that if the motion trackingapparatus 129 detects a movement of an end effector 121 from thepre-determined position and orientation with respect to the patient 103,the surgeon or other clinicians may be promptly notified. Inembodiments, when the motion tracking apparatus 129 detects a change inposition or orientation of the end effector 121 with respect to thepatient 103 by more than a threshold amount, the motion trackingapparatus 129 may send a message to the imaging system 125 and thecontroller 105 of the robotic arm(s) to stop all motion of the system100.

When the gantry 40 is moved such that the patient 103 is located in thebore 16 of the gantry 40, the imaging device 125 may be operated toobtain imaging data of the patient 103 (e.g., a CT scan of at least aportion of the patient 103). The system 100 may therefore be configuredas shown in FIGS. 2A-2C, with the gantry 40 moved over the surgical areaand at least a portion of the robotic arms 101 a, 101 b including theend effectors 121 are located within the bore 16. In embodiments, theend effectors 121 may comprise a radiolucent material so as not to blockx-rays. The updated image data may be shown on the display 111, and mayenable the surgeon to confirm the location of a surgical tool 122inserted into the patient 103. After the image(s) are acquired by theimaging device 125, the gantry 40 may be moved out of the surgical area,such as by translating the gantry 40 to the position as shown in FIGS.1A-1C. The controller 105 of the robotic arms 101 a, 101 b may againcontrol the robotic arms to maintain the predetermined position andorientation of the end effectors 121 with respect to the patient 103while the gantry 40 translates with respect to the patient.

An alternative embodiment of a system 100 for robotically-assistedsurgery is shown in FIGS. 6A-6C and 7A-7C. The system 100 in thisembodiment is substantially identical to the system 100 described abovewith reference to FIGS. 1A-2C. This embodiment differs from theembodiments described above in that there is a single robotic arm 101(rather than the pair of arms 101 a, 101 b shown in FIGS. 1A-2C).

Similar to the embodiment of FIGS. 1A-2C, the robotic arm 101 a and thecamera 131 for the motion tracking system 129 are attached to the gantry40 of the imaging device 125. However, in this embodiment, the roboticarm 101 and the camera 131 are attached to the side of the gantry thatfaces away from the patient 103.

The configuration as shown in FIGS. 6A-6C and 7A-7C may beadvantageously used, for example, for a cranial surgical procedure(e.g., neurosurgery, deep brain stimulation, insertion of an externalventricular drain, etc.). As shown in FIGS. 6A-6C, for example, the head601 of the patient 103 may be stabilized at one end of the patientsupport 60. The head 601 may be located within the bore 16 of the gantry40 of the imaging device 125 for obtaining pre-operative orintra-operative image data. The robotic arm 101 may be moved to aposition such that the end effector 121 is in a predetermined positionand orientation with respect to the patient 103, as described above. Thegantry 40 may be moved down along the length of the patient's body toprovide easier access to the surgical area, as shown in FIGS. 7A-7C. Thecontroller 105 of the robotic arm 101 may control the movements ofrobotic arm 101 such that the end effector 121 is maintained in thepre-determined position and orientation with respect to the patient 103while the gantry 40 moves. As shown in FIGS. 7A-7C, this may include astretching out or unfolding of the arm. Similarly, as the gantry 40translates towards the head 601 of the patient 103 the joints of the arm101 may be folded up as shown in FIGS. 6A-6C. In either case, thecontroller 105 of the robotic arm 101 may use inverse kinematics toensure that the position and orientation of the end effector 121 withrespect to the patient 103 is maintained without any portion of the arm101 colliding with either the imaging device 125 or the patient 103.

As shown in FIGS. 6A-7C, the camera 131 of the motion tracking apparatus129 and/or the arm 135 to which the camera 131 is attached may move inresponse to the movement of the gantry 40 to maintain the surgical areawithin the field-of-view of the camera 131.

Yet another embodiment of a system 100 for robotically-assisted surgeryis shown in FIGS. 8A-8D and 9A-9B. The system 100 in this embodiment issubstantially identical to the system 100 as previously described. Thisembodiment differs from those described above in that a pair of roboticarms 101 a, 101 b are attached to the patient support 60 rather than tothe gantry 40 of the imaging device 125. The camera 131 for the motiontracking system 129 is attached to the gantry 40 of the imaging device125. The robotic arms 101 a, 101 b may be attached at any suitablelocation on the patient support 60, such as on surgical rails 801 thatextend along the sides of the patient support 60. In embodiments, anadaptor 803 may be secured to a surgical rail 801, and the robotic armmay be snapped into or otherwise secured to the adaptor.

The operation of the embodiment shown in FIGS. 8A-8D and 9A-9B may besimilar to the previous embodiments. FIGS. 8A-8D show the gantry 40translated over the surgical area and FIGS. 9A-9B show the gantry 40translated away from the surgical area in order to allow the surgeonaccess to the surgical area. In this embodiment, the robotic arms 101 a,101 b are attached proximate to the distal end of the patient support 60(i.e., opposite the support column 50). Thus, as shown in FIGS. 8A-8Dand 9A-9B, the robotic arms 101 a, 101 b extend into or through the bore16 of the gantry 40 in order to move the end effectors 121 to thepre-determined position and orientation with respect to the patient 103.

A difference between the embodiment shown in FIGS. 8A-8D and theprevious embodiments is that because the robotic arms 101 a, 101 b areattached to the patient support 60 rather than the gantry 40, therobotic arms 101 a, 101 b may not need to move with respect to thepatient 103 once the end effectors 121 are moved to the predeterminedposition and orientation. The controller 105 of the robotic arms 101 a,101 b may move the end effectors 121 to the pre-determined position andorientation such as shown in FIGS. 8A-8D and 9A-9B without colliding thearms 101 a, 101 b with either the imaging system 125 or the patient 103.The arms 101 a, 101 b may be moved to a configuration such that theywill not collide with the gantry 40 as the gantry 40 moves (e.g.,translates) with respect to the patient 103, such as between thepositions shown in FIGS. 8A-8D and 9A-9B, respectively. Alternately, thearms 101 a, 101 b may be moved to an initial configuration with the endeffectors 121 in the pre-determined position and orientation withrespect to the patient 103, and a portion of the arm(s) 101 a, 101 b maybe moved to avoid colliding with the gantry 40 and the patient 103 whilemaintaining the position and orientation of the end effectors 121 whenthe gantry 40 moves with respect to the patient.

In this embodiment, the camera 121 of the motion tracking apparatus 129is attached to the gantry 40 by arm 135. As shown in FIGS. 8A-8D and9A-9B, the camera 131 and/or the arm 135 may move in response to themovement of the gantry 40 to maintain the surgical area within thefield-of-view of the camera 131.

Yet another embodiment of a system 100 for robotically-assisted surgeryis shown in FIGS. 10A-10C and 11A-11C. In this embodiment, the patientsupport 60 is configured for a patient 103 in a seated position. Arobotic arm 101 and a camera 131 for the motion tracking apparatus 129are both mounted to the patient support 60. In this embodiment, therobotic arm 101 and camera 131 are attached to surgical rails 801extending along opposite sides of the patient support 60. FIGS. 10A-10Cshow the gantry 40 of the imaging device 125 translated away from thepatient 103, and FIGS. 11A-11C show the gantry 40 translated to thepatient 103 such that the surgical area is located within the bore 16.The gantry 40 is tilted with respect to the gimbal 30 in thisembodiment.

The operation of the system 100 in the embodiment shown in FIGS. 10A-10Cand 11A-11B may be substantially identical to the embodiments describedabove. The robotic arm 101 may be moved to a position such that the endeffector 121 is in a predetermined position and orientation with respectto the patient 103, as described above. The camera 131 and arm 135 maybe positioned such that the camera 131 is looking into the surgicalarea. If an imaging scan is desired, the gantry 40 which is tilted onthe gimbal 30 may be translated towards the patient 103, such as shownin FIGS. 11A-11C. The controller 105 of the robotic arm 101 may useinverse kinematics to move the robotic arm 101 to maintain the positionand orientation of the end effector 121 with respect to the patient 103without any portion of the arm 101 colliding with either the imagingdevice 125 or the patient 103.

In some embodiments, the patient support 60 may be rotatable withrespect to the imaging device 125, such as shown in FIGS. 12A-12B, whichmay provide the surgeon with additional space for performing a surgicalprocedure. FIGS. 12A-12B show the patient support 60 rotated 90° withrespect to the imaging device 125. When an additional imaging scan isdesired, the patient support 60 may be rotated back to the position asshown in FIGS. 10A-10C, and the gantry 40 may be translated over thepatient 103 to obtain the imaging scan, as shown in FIGS. 11A-11C. Inthis embodiment, the robotic arm 101 is mounted to the patient support60 and thus moves with the patient support 60 as it rotates. In otherembodiments, the robotic arm 101 may be mounted to another structure,such as the imaging device 125 or to a separate support structure, andthe controller 105 of the robotic arm 101 may be configured to move therobotic arm 101 to maintain the end effector 121 in the pre-determinedposition and orientation with respect to the patient 103 as the patientsupport 60 rotates.

As discussed above, the robotic arm 101 may be attached anywhere on thesystem 100, such as the on the gantry 40, the patient support 60, thesupport column 50, the base 20 or the gimbal 30. Mounting a robotic armon the gimbal 30 may enable the robotic arm 101 to remain in closeproximity to the gantry 40 and easily extend into the bore 16 of thegantry 40 without the weight of the robotic arm 101 being distributedonto the gantry 40 itself, which may be weight balanced. In addition,attaching the robotic arm 101 to the gimbal 30 may enable the gantry 40to be tilted with respect to the patient without also tilting the firstend 117 of the robotic arm 101 with respect to the patient. One or morerobotic arms 101 may be mounted directly to the gimbal 30 (e.g.,proximate to the end(s) of one or both of the arms 31,33 that attach toopposing sides of the gantry 40). Alternately, a plate or other supportmember may be attached to and extend from an arm 31, 33 of the gimbal30, and the robotic arm 101 may be mounted to the plate/support member.

In an embodiment shown in FIGS. 13A-13D, a support member 1300 mayextend from the gimbal 30 (e.g., from the end of an arm 31, 33 of thegimbal 30) and at least one robotic arm 101 may be mounted to thesupport member 1300. In embodiments, the support member 1300 may extendat least partially around an outer circumference of the gantry 40. Inthe embodiment of FIGS. 13A-13D, the support member 1300 comprises acurved rail that extends around the outer circumference of the gantry40. In this embodiment, the support member 1300 forms a semicircular arcthat extends between the ends of the respective arms 31 and 33 of thegimbal 30. The semicircular support member 1300 may be concentric withthe outer circumference of the gantry 40.

A bracket mechanism 1301 may be located on the support member 1300 andmay include a mounting surface 1303 for mounting the first end 117 ofthe robotic arm 101 to the bracket mechanism 1301. As shown in FIGS.13A-13D, the mounting surface 1303 may project from the side of thesupport member and may be upwardly angled as shown in FIGS. 13A-13D.This may provide additional clearance for the “tilt” motion of thegantry 40 relative to the gimbal 30.

The bracket mechanism 1301 and the robotic arm 101 attached thereto maybe moved to different positions along the length of support member 1300(e.g., any arbitrary position between the ends of the arms 31, 33 of thegimbal 30) and may be fixed in place at a particular desired positionalong the length of the support member 1300. This is schematicallyillustrated in FIGS. 13C and 13D which are perspective and front viewsof the system 100 illustrating the bracket mechanism 1301 and therobotic arm 101 in a first position and a second position (indicated byphantom) on the support member 1300. In some embodiments, the bracketmechanism 1301 may be moved manually (e.g., positioned by an operator ata particular location along the length of the support member 1301 andthen clamped or otherwise fastened in place). Alternately, the bracketmechanism 1301 may be automatically driven to different positions usinga suitable drive mechanism (e.g., a motorized belt drive, frictionwheel, gear tooth assembly, cable-pulley system, etc., not shown inFIGS. 13A-13D). The drive mechanism may be located on the bracketmechanism 1301, the support member 1300 and/or the gimbal 30, forexample. An encoder mechanism may be utilized to indicate the positionof the bracket mechanism 1301 and the first end 117 of the robotic arm101 on the support member 1300. Although the embodiment of FIGS. 13A-13Dillustrates one robotic arm 101 mounted to the support member 1300, itwill be understood that more than one robotic arm may be mounted to thesupport member 1300 via respective bracket mechanisms 1301.

FIG. 13A-13D also illustrates a motion tracking apparatus 1305 which issimilar to the motion tracking apparatus described above. In thisembodiment, the motion tracking apparatus 1305 includes a stereoscopicoptical sensor device 1304 that includes two or more IR cameras 1306,1308 attached to the gantry 40 of the imaging device. The optical sensordevice 1304 may include one or more IR sources (e.g., diode ring(s))that direct IR radiation into the surgical field, where the IR radiationmay be reflected by markers and received by the cameras 1306, 1308. Asshown in FIG. 13 A, a plurality of markers 1307 may be attached to thepatient to form a “reference arc” that enables the cameras 1306, 1308 totrack the patient. The optical sensor device 1305 may include a two-axisrobotic system to enable the cameras 1306, 1308 to tilt and pan (i.e.,rotate up-and-down and side-to-side) such that the reference arc on thepatient may be maintained in the center of the cameras' field of view. Asecond set of markers 1309 may be attached to the second end 119 of therobotic arm 101 or to the end effector 121 to enable the tracking systemto track the end effector 121. The second set of markers 1309 preferablycomprises four or more non-coplanar markers in a fixed, known geometricrelationship with each other and to the end effector 121, which enablesboth the position (x, y, z) and the orientation (yaw, pitch, roll) ofthe end effector 121 to be fully resolved. Similar sets of markers maybe provided on any instruments or other objects brought into thesurgical field to allow these objects to be tracked.

In the embodiment of FIGS. 13A-13D, the optical sensor device 1304 ismounted to the first (i.e., front) side 127 of the gantry 40 via asecond support member 1311. The second support member 1311 may be acurved (e.g., semicircular) rail that may be attached to an upperportion of the gantry 40 to enable the cameras 1306, 1308 to look downinto the surgical field. The optical sensor device 1304 may be movableto different positions along the second support member 1311, eithermanually or using a drive system. This may provide flexibility so thatthe robotic arm 101 may be translated to any location on support member1300 while the optical sensor device 1304 may be translated to one sideor the other of the robotic arm axis so that the cameras 1306, 1308 mayremain pointed down into the surgical field without being occluded bythe end of the robotic arm 101. Other motion tracking apparatuses, suchas the apparatus 129 described above with reference to FIGS. 1A-12B,could be utilized in the system of FIGS. 13A-13D.

FIGS. 14A-14C illustrate an alternative embodiment of a system 100 forperforming robotically-assisted surgery that includes at least onerobotic arm 101 mounted to an imaging system 1400. In this embodiment,the imaging system 1400 includes an O-shaped imaging gantry 1401 that ismounted to a support structure 1403 in a cantilevered fashion. Theimaging system 1400 may be an x-ray imaging system that may be used toobtain 2D fluoroscopic images and/or 3D tomographic images of an objectlocated within the bore 16 of the gantry. At least one of an x-raysource and an x-ray detector (not visible in FIGS. 14A-14C) may rotatearound the interior of the gantry 1401 to obtain images of an objectwithin the bore 16 from different projection angles. The supportstructure 1403 may comprise a mobile cart 1406 that is attached to oneside of the gantry 1401 via an attachment mechanism 1405. The attachmentmechanism 1405 may include one or more motorized systems that enable thegantry 1401 to translate and/or rotate with respect to at least aportion of the cart 1406. For example, in embodiments the gantry 1401may be raised or lowered relative to the cart 1406 and/or may betranslated over a limited range-of-motion along the z-axis (i.e., intoand out of the page in FIG. 14A) relative to the cart 1406. In addition,in some embodiments the gantry 1401 may be rotated with respect to thecart 1406 along one or more axis. For example, the gantry 1401 may betilted with respect to the cart 1406 about a horizontal axis extendingthrough the attachment point between the gantry 1401 and cart 1406and/or may have a “wag” rotation about a vertical axis with respect tothe cart 1406.

One or more robotic arms 101 may be attached anywhere on the imagingsystem 1400 of FIGS. 14A-14C, such as the on the gantry 1401, the cart1406 or the attachment mechanism 1405. In an embodiment shown in FIGS.14A-14C, the robotic arm 101 is attached to a support member 1407 whichmay be similar to the support member 1300 described above with referenceto FIGS. 13A-13D. In this embodiment, the support member 1407 extendsfrom the attachment mechanism 1405, although the support member 1407 mayextend from any portion of the cart 1406. The support member 1407 inthis embodiment is a semicircular segment that extends concentricallyover an upper portion of the gantry 1401. The support member 1407 andthe robotic arm 101 secured thereto may translate with the translationof the gantry 1401 along at least one axis (e.g., up and downtranslation) relative to the cart 1406. In embodiments, the gantry 1401may be able to rotate (e.g., tilt) with respect to the cart 1406 withoutthe support member 1407 and robotic arm 101 also rotating.

The robotic arm 101 may be attached to the support member 1407 using abracket mechanism 1301 as described above. The bracket mechanism 1301and robotic arm may be moved to any arbitrary position along the lengthof the support member 1407. In addition, the system may include atracking system comprising an optical sensor device 1304 mounted to aside of the gantry 1401 via a second support member 1313, as isdescribed above with reference to FIGS. 13A-13D. The optical sensordevice 1304 may be moveable to different positions along the length ofthe second support member 1313, as described above. Other motiontracking apparatuses, such as the apparatus 129 described above withreference to FIGS. 1A-12B, could be utilized in the system of FIGS.14A-14C.

FIGS. 15A-15D illustrate another alternative embodiment of a system 100for performing robotically-assisted surgery that includes at least onerobotic arm 101 mounted to an imaging system 1500. In this embodiment,the imaging system 1500 is a C-arm device that includes an x-ray source1501 and a detector 1503 connected to one another by a C-shapedconnecting member 1505. The C-shaped connecting member 1505 is coupledto a support structure 1507, which in this embodiment comprises a mobilecart 1509. An attachment mechanism 1511 attaches the C-shaped connectingmember 1505 to the cart 1509 such that the attachment mechanism 1511together with the source 1501, detector 1503 and C-shaped connectingmember 1505 may be rotated in a first direction (i.e., into and out ofthe page in FIG. 15A) relative to the cart 1509. In some embodiments,the source 1501, detector 1503 and C-shaped connecting member 1505 mayalso rotate in a second direction (i.e., within the plane of the page inFIG. 15A) relative to the attachment mechanism 1511 and the cart 1509.As discussed above, the cart 1509 may be a mobile cart and may be usedto move the entire imaging system 1500 to a desired position andorientation. The source 1501 and detector 1503 may be used to obtainx-ray images, such as 2D fluoroscopic images, of an object positionedbetween the source 1501 and detector 1503 from a variety of differentprojection angles.

One or more robotic arms 101 may be attached anywhere on the imagingsystem 1500 of FIGS. 15A-15D, such as the on the cart 1509, theattachment mechanism 1511 or the C-shaped connecting member 1505. In anembodiment shown in FIGS. 15A-15D, the robotic arm 101 is attached to asupport member 1513 which may be similar to the support members 1300 and1407 described above with reference to FIGS. 13A-13D and 14A-14C. Inthis embodiment, the support member 1513 extends from the cart 1509.

The support member 1513 in this embodiment is a curved (e.g.,semicircular) segment that extends from the cart 1509 at least partiallyabove the source 1501, detector 1503 and the C-shaped connecting member1505. The support member 1513 may be concentric with the C-shapedconnecting member 1505. The support member 1513 may be located such thatthe source 1501 and detector 1503 may freely rotate about one or moreaxes without contacting the support member 1513.

The robotic arm 101 may be attached to the support member 1513 using abracket mechanism 1301 as described above. The bracket mechanism 1301and robotic arm 101 may be moved to any arbitrary position along thelength of the support member 1513. In addition, the system may include atracking system comprising an optical sensor device 1304 mounted to aside of the gantry 1401 via a second support member 1515. The secondsupport member 1515 may be a second curved (e.g., semicircular) segmentthat extends from the cart 1509 at least partially above the source1501, detector 1503 and the C-shaped connecting member 1505. The secondsupport member 1515 may extend parallel to support member 1513, as shownin FIGS. 15A-15C. In this embodiment, the second support member 1515extends for a shorter length than support member 1513, although it willbe understood that the second support member 1515 may extend for thesame or a greater length than support member 1513. The optical sensordevice 1304 may be moveable to different positions along the length ofthe second support member 1515, as described above. Alternately, boththe robotic arm 101 and the optical sensor device 1304 may be mounted tothe same support member (e.g., support member 1513). Also, other motiontracking apparatuses, such as the apparatus 129 described above withreference to FIGS. 1A-12B, could be utilized in the system of FIGS.15A-15D.

FIG. 16 is a system block diagram of a computing device useful toperform functions of a processing control unit, such as controllers 105,205 and 213 described above. While the computing device 1600 isillustrated as a laptop computer, a computing device providing thefunctional capabilities of the computer device 1600 may be implementedas a workstation computer, an embedded computer, a desktop computer or ahandheld computer (e.g., tablet, a smartphone, etc.). A typicalcomputing device may include a processor 1601 coupled to an electronicdisplay 1604, a speaker 1606 and a memory 1602, which may be a volatilememory as well as a nonvolatile memory (e.g., a disk drive). Whenimplemented as a laptop computer or desktop computer, the computingdevice 1600 may also include a floppy disc drive, compact disc (CD) orDVD disc drive coupled to the processor 1601. The computing device 1600may include an antenna 1610, a multimedia receiver 1612, a transceiver1618 and/or communications circuitry coupled to the processor 1601 forsending and receiving electromagnetic radiation, connecting to awireless data link, and receiving data. Additionally, the computingdevice 1600 may include network access ports 1624 coupled to theprocessor for establishing data connections with a network (e.g., LANcoupled to a service provider network, etc.). A laptop computer ordesktop computer 1600 typically also includes a keyboard 1614 and amouse pad 1616 for receiving user inputs.

The foregoing method descriptions are provided merely as illustrativeexamples and are not intended to require or imply that the steps of thevarious embodiments must be performed in the order presented. As will beappreciated by one of skill in the art the order of steps in theforegoing embodiments may be performed in any order. Words such as“thereafter,” “then,” “next,” etc. are not necessarily intended to limitthe order of the steps; these words may be used to guide the readerthrough the description of the methods. Further, any reference to claimelements in the singular, for example, using the articles “a,” “an” or“the” is not to be construed as limiting the element to the singular.

The various illustrative logical blocks, modules, circuits, andalgorithm steps described in connection with the embodiments disclosedherein may be implemented as electronic hardware, computer software, orcombinations of both. To clearly illustrate this interchangeability ofhardware and software, various illustrative components, blocks, modules,circuits, and steps have been described above generally in terms oftheir functionality. Whether such functionality is implemented ashardware or software depends upon the particular application and designconstraints imposed on the overall system. Skilled artisans mayimplement the described functionality in varying ways for eachparticular application, but such implementation decisions should not beinterpreted as causing a departure from the scope of the presentinvention.

The hardware used to implement the various illustrative logics, logicalblocks, modules, and circuits described in connection with the aspectsdisclosed herein may be implemented or performed with a general purposeprocessor, a digital signal processor (DSP), an application specificintegrated circuit (ASIC), a field programmable gate array (FPGA) orother programmable logic device, discrete gate or transistor logic,discrete hardware components, or any combination thereof designed toperform the functions described herein. A general-purpose processor maybe a microprocessor, but, in the alternative, the processor may be anyconventional processor, controller, microcontroller, or state machine. Aprocessor may also be implemented as a combination of computing devices,e.g., a combination of a DSP and a microprocessor, a plurality ofmicroprocessors, one or more microprocessors in conjunction with a DSPcore, or any other such configuration. Alternatively, some steps ormethods may be performed by circuitry that is specific to a givenfunction.

In one or more exemplary aspects, the functions described may beimplemented in hardware, software, firmware, or any combination thereof.If implemented in software, the functions may be stored on as one ormore instructions or code on a non-transitory computer-readable medium.The steps of a method or algorithm disclosed herein may be embodied in aprocessor-executable software module executed which may reside on anon-transitory computer-readable medium. Non-transitorycomputer-readable media includes computer storage media that facilitatestransfer of a computer program from one place to another. A storagemedia may be any available media that may be accessed by a computer. Byway of example, and not limitation, such non-transitorycomputer-readable storage media may comprise RAM, ROM, EEPROM, CD-ROM orother optical disk storage, magnetic disk storage or other magneticstorage devices, or any other medium that may be used to carry or storedesired program code in the form of instructions or data structures andthat may be accessed by a computer. Disk and disc, as used herein,includes compact disc (CD), laser disc, optical disc, digital versatiledisc (DVD), floppy disk, and blu-ray disc where disks usually reproducedata magnetically, while discs reproduce data optically with lasers.Combinations of the above should also be included within the scope ofnon-transitory computer-readable storage media. Additionally, theoperations of a method or algorithm may reside as one or any combinationor set of codes and/or instructions on a machine readable medium and/orcomputer-readable medium, which may be incorporated into a computerprogram product.

The preceding description of the disclosed aspects is provided to enableany person skilled in the art to make or use the present invention.Various modifications to these aspects will be readily apparent to thoseskilled in the art, and the generic principles defined herein may beapplied to other aspects without departing from the scope of theinvention. Thus, the present invention is not intended to be limited tothe aspects shown herein but is to be accorded the widest scopeconsistent with the principles and novel features disclosed herein.

What is claimed as new and desired to be protected by Letters Patent ofthe United States is:
 1. A method of performing robotically-assistedsurgery, comprising: moving an end effector and a robotic armoperatively attached with an imaging device with respect to a patientand the imaging device to a pre-determined position and orientation withrespect to the patient based on imaging data of the patient obtained bythe imaging device; moving the imaging device and the robotic armrelative to the patient by effecting linear translation along a base ofthe imaging device, moving the imaging device and the robotic armlinearly relative to the patient; detecting a movement of the imagingdevice relative to the patient; and obtaining an intraoperative image ofthe patient while moving the robotic arm, avoiding collisions with theimaging device and with the patient in response to detecting a movementof the imaging device relative to the patient.
 2. The method of claim 1,wherein the robotic arm comprises a multijoint arm and is moved usinginverse kinematics.
 3. The method of claim 1, wherein the end effectoris configured to receive and guide an invasive surgical tool.
 4. Themethod of claim 1, wherein the pre-determined position and orientationof the end effector with respect to the patient is determined based on auser selection of an entrance point and a target point in a display ofthe imaging data of the patient obtained by the imaging device.
 5. Themethod of claim 1, wherein the imaging device comprises a gantrycontaining at least one imaging component and defining a bore, theimaging component including an x-ray computed tomography (CT) scanningdevice.
 6. The method of claim 5, wherein a support column extends abovethe base, and a patient support is mounted to the support column,wherein the gantry is located above the base; and wherein the gantrytranslates and tilts relative to the base.
 7. The method of claim 1,further comprising: tracking a position of at least one of the roboticarm and the imaging device using a motion tracking apparatus; whereinthe motion tracking apparatus comprises at least one marker fixed to therobotic arm and the imaging device and a sensing device that detectsradiation emitted or reflected by the at least one marker; wherein thesensing device is attached to the imaging device and moves independentlyof the movement of the imaging device to maintain a surgical area of thepatient within a field of view of the sensing device; and wherein themotion tracking apparatus is configured to detect a movement of the endeffector from the pre-determined position and orientation with respectto the patient.
 8. The method of claim 7, wherein the position of therobotic arm with respect to the patient and the imaging device isdetermined based on position data received from at least one of themotion tracking apparatus and the imaging device; and wherein therobotic arm, the imaging device and the motion tracking apparatusoperate in a common coordinate system.
 9. The method of claim 7, furthercomprising generating a boundary surface encompassing at least a portionof the patient, wherein movements of the robotic arm are controlled suchthat no portion of the robotic arm may enter the boundary surface; andwherein the boundary surface is generated by tracking a plurality ofmarkers on the patient using the motion tracking apparatus.
 10. Themethod of claim 9 further comprising at least one of notifying a userand stopping the motion of the imaging device in response to detecting amovement of the end effector from the pre-determined position andorientation with respect to the patient.
 11. The method of claim 1,further comprising: determining that there are no movements of therobotic arm that would not result in either changing the position ororientation of the end effector with respect to the patient or collidingwith the imaging device or the patient, and based on this determination,performing operations comprising at least one of: issuing an alert to auser; and stopping motion of the imaging device with respect to thepatient.
 12. A system for performing robotically-assisted surgery,comprising: a robotic arm having an arm base and an end effector; animaging device including a base and a gantry, the gantry is connectedwith the arm base and translates along the base and a length of apatient to obtain imaging data of the patient; and a processor coupledto the robotic arm and configured with processor executable instructionsto perform operations comprising: controlling the robotic arm to movethe end effector to a pre-determined position and orientation withrespect to the patient based on imaging data of the patient obtained bythe imaging device; detecting a movement of the imaging device relativeto the patient; and moving the robotic arm while avoiding collisionswith the imaging device and with the patient in response to detecting amovement of the imaging device relative to the patient during anintraoperative scan.
 13. The system of claim 12 further comprising apatient support, and the imaging device linearly translates along thebase and a length of the patient support to obtain imaging data of apatient positioned on the patient support.
 14. The system of claim 13further comprising a motion tracking apparatus comprising a cameraoperatively attached to the imaging device that is configured to trackobjects in a surgical area, including at least the patient and therobotic arm; and wherein the camera moves relative to the patientsupport to maintain the surgical area in a field of view of the camera.15. A system for performing robotically-assisted surgery, comprising: anx-ray imaging device comprising a base and a support structure, thesupport structure configured to linearly translate along the base, thesupport structure including an x-ray source and an x-ray detectormounted to the support structure such that at least one of the x-raysource and the x-ray detector is configured to rotate with respect tothe support structure to obtain x-ray images from different projectionangles relative to an object being imaged; and a robotic arm having afirst end configured to extend into an imaging area between the x-raysource and the x-ray detector and a second end attached to the supportstructure; wherein the at least one of the x-ray source and the x-raydetector is configured to rotate around two mutually perpendicular axeswith respect to the support structure and the second end of the roboticarm; and wherein the robotic arm moves with the support structurerelative to the base when the support structure moves to obtain x-rayimages while the x-ray imaging device obtains x-ray images fromdifferent projection angles relative to the object being imagined. 16.The system of claim 15, wherein the at least one of the x-ray source andthe x-ray detector is configured to translate in at least one directionwith respect to a first portion of the support structure, and the secondend of the robotic arm is attached to a second portion of the supportstructure that translates with the at least one of the x-ray source andthe x-ray detector relative to the first portion of the supportstructure.
 17. The system of claim 16, further comprising: a firstsupport member extending from the support structure over at least aportion of an outer circumference of an imaging gantry containing thex-ray source and the x-ray detector, wherein the second end of therobotic arm is mounted to the first support member and is movable tomultiple positions on the first support member; and a second supportmember attached to the imaging gantry, wherein at least one camera for amotion tracking system is mounted to the second support member and ismovable to multiple positions on the second support member.
 18. Thesystem of claim 15, wherein the imaging device comprises an O-shapedimaging gantry containing the x-ray source and the x-ray detector andthe O-shaped imaging gantry is configured to tilt with respect to thesupport structure and the second end of the robotic arm.
 19. The systemof claim 18, wherein the support structure comprises a gimbal having apair of arms that connect to the imaging gantry on opposite sides of thegantry, and the second end of the robotic arm is attached to the gimbal.20. The system of claim 19, wherein the second end of the robotic arm isattached to a support member that extends from an end of an arm of thegimbal and above at least a portion of an outer circumference of theO-shaped imaging gantry.